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Introduction 


Control  systems  for  complex  man i p u lator u tasks  require  pre- 
cise- information  of  errors  in  position,  orientation,  and  velocity 
of  the  manipulator  hand  with  respect  to  the  object  to  be  manipu- 
lated. This  is  particularly  true  in  the  final  approach  trajecto- 
ry just  prior  to  contact  between  the  hand  and  object.  In  order 
to  provide  this  information  in  a form  usable  by  a human  operator 
or  a computer  control  system  (or  both)  the  National  Engineering 
Laboratory  of  the  National  Bureau  of  Standards  has  developed  for 
Marshall  Space  Flight  Center  a Proximity-Vision  System  suitably 
configured  to  mount  on  the  MSFC  Protoflight  Manipulator  Arm 
(PFMA).  This  Proximity-Vision  System  is  a breadboard  system 
built  in  order  to  demonstrate  the  capabilities  and  analyze  the 
problems  associated  with  using  such  a system  in  a real-time  con- 
trol mode.  This  report  describes  the  theory  of  operation  and  the 
physical  and  electrical  components  of  the  breadboard  hardware 
delivered  to  Marshall  Space  Flight  Center.  Performance  testing 
of  this  hardware  is  to  be  done  by  the  recipient. 

Technical  Approach 

A block  diagram  of  the  NBS  Proximity-Vision  System  is  shown 
in  Figure  1.  This  system  consists  of  two  separate  but  complemen- 
tary subsystems. 


1.  A solid-state  TV  camera  with  128  x 128  resolution  ele- 
ments is  mounted  on  the  wrist  of  the  PFMA  just  behind  the 
fingers,  as  illustrated  in  Figure  2.  This  camera  is  positioned 
so  that  its  36  degree  field  of  view  covers  the  manipulator 
fingers  and  a region  extending  one  meter  in  front  of  the  fingers, 
A photograph  of  the  camera  is  shown  in  Figure  3. 


Coordinated  with  this  camera  is  a high  intensity  strobe 
flash  system  with  optics  which  projects  a thin  fan-shaped  plane 
of  light  into  the  region  viewed  by  the  camera  in  front  of  the 
PFMA  fingers.  A photograph  of  the  flash  system  is  shown  in  Fig- 
ure 4.  Any  diffusely  reflecting  object  within  the  region  in 
front  of  the  fingers  reflects  the  light  beam.  The  apparent  posi- 
tion in  the  camera  field  of  view  of  the  reflected  light  makes  it 
possible  to  compute  the  distance  of  the  reflecting  object  by  sim- 
ple tr iang u la t i on.  The  offset  between  the  camera  and  the  plane 
of  projected  light  d etermineSth e relationships  between  the  dis- 
tance and  the  apparent  position  of  the  reflected  light. 


The  fact  that  the  projected  light  is  a plane  means  that  the 
intersection  of  the  plane  with  a solid  object  will  produce  a re- 
flected line  of  light  as  shown  in  Figure  5.  The  position  of  this 
line  indicates  distance.  Its  shape  conveys  information  concern- 
ing the  depth  profile  of  the  object  being  illuminated,  and  its 
end-points  indicate  the  position  and  spacing  of  the  edges  to  be 
grasped.  This  is  illustrated  in  Figure  6 and  7. 
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2.  Ihe  second  subsystem  of  the  NBS  Proximity-Vision  System 
consists  of  a pair  of  close-range  infra-red  proximity  sensors  im- 
bedded in  the  tips  of  a pair  of  fingers  suitable  for  mounting  on 
the  PFMA.  A photograph  of  the  fingers  is  shown  in  Figure  8.  Two 
light  emitting  diodes/  one  on  each  finger*  project  chopped 
infra-red  light  into  the  region  immediately  in  front  of  the 
fingers.  Two  photo-transistors*  also  one  on  each  finger*  collect 
light  reflected  from  any  diffusely  reflecting  object  immediately 
in  front  of  or  between  the  fingers.  This  is  illustrated  in  Fig- 
ure 9.  The  modulated  light  is  synchronously  detected  in  order  to 
discriminate  against  background  illumination. 


These  short-range  detectors  are  sensitive  in  the  range  of  O 
to  20  centimeters  and  measure  only  the  strength  of  the  reflected 
light.  Since  the  albedo  of  the  reflecting  object  influences  the 
strength  of  the  reflected  infra-red  signal*  this  subsystem 
provides  only  relative  position  information  unless  the  signals 
are  calibrated  in  the  region  when  both  the  solid-state  camera- 
flash  system  and  the  short  range  infra-red  system  overlap. 


These  short  range  detectors  provide  a measure  of  redundancy 
to  the  system*  particularly  in  the  last  few  centimeters  before 
contact.  They  ^Iso  provide  centering  information  for  large  ob- 
jects which  overflow  the  field  of  view  of  the  camera-flash  system 
at  very  close  range. 


Display-Control  Console 


Also  included  in  the  Proximity-Vision  System  is  an  operator 
display-control  console.  A TV  viewing  monitor  displays  the  video 
signal  from  the  solid  state  TV  camera.  Brightness  controls  on 
the  monitor  and  push  buttons  for  remote  adjustment  of  the  "f" 
stop  on  the  camera  enable  the  operator  to  optimize  picture  quali- 
ty. The  strobe  flash  is  syncronized  with  the  camera  frame  rate 
so  that  the  reflected  line  of  light  appears  as  a pulsing  cursor 
superimposed  on  the  image.  There  is  a selector  switch  which  en- 
ables the  operator  to  choose  the  pulse  rate  over  the  range  from  3 
per  second  to  one  every  1. 5 seconds. 


Data  from  the  infra-red  system  is  displayed  on  the  same  mon- 
itor as  the  128  x 128  camera  data.  Intensity  of  reflected  light 
from  the  two  sensors  is  displayed  as  lengths  of  two  bright  lines 
running  up  the  right  and  left  sides  of  the  camera  display  raster. 
The  length  of  each  line  is  proportional  to  the  intensity  of  the 
signal  detected  by  the  respective  sensor. 
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Computer  Interface 


The  display-control  console  also  contains  a computer  inter- 
face which  can  be  used  to  interface  the  camera-flash  system  with 
a coniputer. 


The  video  signal  from  the  TV  camera  is  converted  to  a series 
of  8 bit  values  by  an  analog  to  digital  converter  in  the  control 
console.  Synchronization  with  the  computer  is  provided  via  a 
data  accepted  pulse  from  the  computer.  When  in  the  computer  mode 
(which  is  selected  by  a switch  on  the  front  panel  of  the  control 
console)  the  video  clock  is  interrupted  after  each  A/D  conversion 
until  the  computer  reads  the  8 bit  word  and  sends  a data  accepted 
pulse. 


Conclusions  and  Recommendations 


A full  series  of  operational  tests  of  the  NBS  Proximity- 
Vision  system  with  the  MSFC  Protoflight  Manipulator  Arm  remains 
to  be  done  under  the  auspices  of  MSFC  personnel.  However/  prel- 
iminary tests  performed  prior  to  and  during  delivery  and  instal- 
lation of  the  system  on  the  PFMA  suggest  several  areas  of  further 
work. 


1.  Picture  Quality 


a)  The  intensity  of  the  flash  should  be  controlled  so 
that  it  can  be  increased  for  large  distances  and  decreased  at 
short  range.  Preferably  this  should  be  accomplished  automatical- 
ly. This  would  best  be  done  with  a microprocessor  which  measures 
the  received  signal  and  adjusts  the  strobe  intensity  accordingly. 
The  strobe  intensity  is  best  controlled  by  varying  the  duration 
of  the  flash. 


b)  The  flash  repetition  rate  should  also  be  varied  so 
that  it  is  inversely  proportional  to  distance.  Thus  as  the  mani- 
pulator closes  on  an  object  and  the  intensity  of  the  strobe  de- 
creases/ the  rate  of  the  flashing  strobe  would  increase.  This 
could  also  be  accomplished  by  a microprocessor. 


c)  The  iris  adjustment  to  optimize  the  picture  quality 
should  also  be  placed  under  microprocessor  control. 


2.  Slip  Ring  Requirements 


The  PFMA  has  a continuous  wrist— roll  capability  such  that  the 
electrical  connections  between  the  op  era tor— d i sp lay  control  sta- 
tion and  the  manipulator  wrist  must  pass  through  a set  of  slip 
rings.  The  NBS  Proximity-Vision  system  in  its  present  breadboard 
form  has  a total  of  23  leads  which  cross  the  continuous  wrist- 
roll  interface.  Because  it  is  a breadboard  only  no  serious  at- 
tempt was  made  to  minimize  the  number  of  leads  or  to  operate 
through  the  existing  slip  rings. 


Future  versions  of  the  Proximity-Vision  system  will  need  to 
address  this  Slip  ring  problem.  It  would  be  possible  to  reduce 
the  required  number  of  slip  rings  to  17  by  allowing  several  of 
the  subsystems  to  share  power  supply  voltages.  This  would  re- 
quire additional  filtering  of  supply  voltages  to  prevent  cross 
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talk.  Additional  savings  in  slip  ring  requirements  might  be  made 
by  eliminating  the  End-of-Line  and  End-of-Frame  sync  signals  and 
by  putting  the  video  clock  generator  on  the  wrist.  This  would 
require  the  design  of  additional  circuitry  to  reconstruct  these 
sync  signals  from  the  composite  video. 

The  absolute  minimum  number  of  slip  ring  connections  are  as 
follows: 


Camera 


1. 

+ 15 

2. 

-15 

3. 

GND 

4. 

Video 

5. 

Strobe 

Flash 


6.  +300  volts  (or  +5  volts  with  3 amps  capacity) 

7.  GND 

8.  Trigger 


IR  Sensor s-Emi t ter 5 


9. 

Output 

#1 

10. 

Output 

#2 

11. 

+5  volts 

12. 

GND 

Iris 

Adjust 

13. 

Motor 

Winding  + 

14. 

Motor 

Wind ing  — 

3.  Computer  Analysis  of  Proximity-Vision  Data 


It  is  recommended  that  a continued  development  effort  be  un- 
dertaken to  make  possible  the  automatic  extraction  of  range  and 
position  data  from  the  proximity  vision  system.  This  would  make 
possible  computer  assisted  terminal  guidance  and  control  of  the 
manipulator  in  grasping  objects*  extracting  modules*  and  perform- 
ing various  other  tasks  which  will  be  required  of  the  PFMA  and 
similar  systems  in  the  future.  The  advantages  would  be  increased 
precision*  speed*  and  dexterity  of  terminal  movements*  reduced 
operator  fatigue*  and  improved  reliability  and  safety. 
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As  a first  step  toward  this  goal#  NBS  recommends: 


a)  A microcomputer  system  for  automatic  image  quality 
and  flash  intensity  and  rate  control.  It  is  recommended  that 
this  be  a separate  microcomputer  from  the  PDP-11/35  which  will 
control  the  arm  because  of  the  unique  demands  of  the  video  in- 
terface. Although  it  would  be  possible  to  connect  the 
Proximity-Vision  system  directly  to  the  arm  control  computer 
through  a suitably  configured  interface*  this  is  not  recommended 
because  of  the  potential  software  problems  due  to  conflicting 
timing  requirements  of  the  two  real  time  control  systems. 


b>  A hardware  f rame-to-f rame  compare  and  threshold 
system  so  that  the  line  of  light  produced  by  the  flash  can  be  au- 
tomatically separated  -from  the  background  image.  A block  diagram 
of  such  a system  is  shown  in  Figure  10. 


c)  Software  for  feature  extraction  so  as  to  automati- 
cally measure  target  parameters  such  as: 
o distance 
o closure  rate 
o s i z e 

o orientation 
o depth  profile 

A block  diagram  of  the  recommended  computer  analysis  system 
is  shown  in  Figure  11. 
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FIGURES 


FIGURE  1;  A block  diagram  of  Proximity  Vision  System. 

FIGURE  2;  Mounting  configuration  of  the  Proximity  Vision  System. 

FIGURE  3:  The  128  X 128  solid  state  TV  camera  with  mounting 
bracket  and  iris  adjust  mechanism. 

FIGURE  4:  The  flash  head  which  generates  the  plane  of  light. 

FIGURE  5;  Relationship  between  the  camera  field  of  viewi  the 
plane  of  light/  and  a viewed  object. 

FIGURE  6:  A range  graticule  which  when  placed  over  the  viewing 

monitor  gives  the  distance  to  any  point  on  a reflected  line  of 
light.  Scale  in  inches. 

FIGURE  7:  Illustrations  of  various  shaped  objects  viewed  through 

the  monitor.  The  reflected  line  of  light  is  shown  as  a heavy 
black  line  across  each  object. 

FIGURE  8:  PFMA  finger  tips  containing  the  close-range  infra-red 
proximity  sensors. 

FIGURE  9:  A drawing  illustrating  the  relative  position  of  the 

light  emitting  diodes  (LEDs)  and  proximity  sensors. 

FIGURE  10:  Block  diagram  of  a hardware  f rame-t o-f rame  compare 

system  for  automatic  extraction  of  the  line  of  light  from  back- 
ground illumination. 

FIGURE  11:  Block  diagram  of  recommended  system  for  automatic  con- 

trol of  image  quality  and  extraction  of  target  features. 
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FlflllRE  i:  The  128  X 128  Rolld  state  TV  camera  with  mounting  bracket  and  iris  adjust  mechanism. 
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FIGURE  4:  The  flash  head  which  generates  the  plane  of  light 
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FIGURE  6,  A range  graticule  which  when  placed  over  the  viewing  monitor  gives 

the  distance  to  any  point  on  a reflected  line  of  light.  The  X and  Y 
distances  are  measured  in  the  coordinate  system  of  the  fingers.  The  X 
axis  passes  through  the  two  finger  tips  and  the  Y axis  is  parallel  to 
the  wrist  axis.  The  slight  tile  in  the  figure  is  due  to  a misalignment 
of  the  chip  in  the  camera. 
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figure  7:  Illustrations  of  various  shaped  objects  viewed  through  the 

monitor.  The  reflected  line  of  light  is  shown  as  a heavey 
black  line  across  each  object. 


FIGURE  8,:  PFMA  finger  tips  containing  the  close-range  infra-red  proximity  senso 
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FIGURE  9:  A drawing  illustrating  the  relative  position  of  the  light  emitting  diodes  (LEDs)  and 

proximity  sensors. 


CCD  FRAME  BUFFER 


c 

o 


CO 

E 

o 

4-1 

3 

CO 

M 

O 

CM 

E 

OJ 

4-J 

CO 

CO 

CD  • 
Fh  C 
CO  O 
CX  *H 

E M 
O CO 

0 C 

04  E 
E 3 

CO  rH 
M r-l 
<M  -H 

1 

O TO 
M C 


CO  00 
J-l 

<M  a 
cO 

Oi  ^ 


M 

CO 

S 

TO 


M C|_i 

CO 

x;  4-1 
-C 
CO  CjO 

•H 

4h  r-H 

O 

CM 

E O 
CO 

M 0) 
00  C ' 
cO  -H 
•H  I — I 
TO 

CU 
XI 
O 4-) 

O 

rM  CM 
P3  O 


o 

1 — I 

w 


C3 

M 

Pc-i 


FIGURE  11 : Block  diagram  of  recommended  system  for  automatic  control  of  image  quality  and 
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